REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
نویسندگان
چکیده
Learning in models with discrete latent variables is challenging due to high variance gradient estimators. Generally, approaches have relied on control variates to reduce the variance of the REINFORCE estimator. Recent work (Jang et al., 2016; Maddison et al., 2016) has taken a different approach, introducing a continuous relaxation of discrete variables to produce low-variance, but biased, gradient estimates. In this work, we combine the two approaches through a novel control variate that produces low-variance, unbiased gradient estimates. We present encouraging results on a toy problem and on learning sigmoid belief networks.
منابع مشابه
Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
Gradient-based optimization is the foundation of deep learning and reinforcement learning, but is difficult to apply when the mechanism being optimized is unknown or not differentiable. We introduce a general framework for learning low-variance, unbiased gradient estimators, applicable to black-box functions of discrete or continuous random variables. Our method uses gradients of a surrogate ne...
متن کاملNeural Variational Inference and Learning in Belief Networks
•We introduce a simple, efficient, and general method for training directed latent variable models. – Can handle both discrete and continuous latent variables. – Easy to apply – requires no model-specific derivations. •Key idea: Train an auxiliary neural network to perform inference in the model of interest by optimizing the variational bound. – Was considered before for Helmholtz machines and ...
متن کاملMuProp: Unbiased Backpropagation for Stochastic Neural Networks
Deep neural networks are powerful parametric models that can be trained efficiently using the backpropagation algorithm. Stochastic neural networks combine the power of large parametric functions with that of graphical models, which makes it possible to learn very complex distributions. However, as backpropagation is not directly applicable to stochastic networks that include discrete sampling ...
متن کاملVariational Inference for Monte Carlo Objectives
Recent progress in deep latent variable models has largely been driven by the development of flexible and scalable variational inference methods. Variational training of this type involves maximizing a lower bound on the log-likelihood, using samples from the variational posterior to compute the required gradients. Recently, Burda et al. (2015) have derived a tighter lower bound using a multi-s...
متن کاملStochastic Backpropagation through Mixture Density Distributions
The ability to backpropagate stochastic gradients through continuous latent distributions has been crucial to the emergence of variational autoencoders [4, 6, 7, 3] and stochastic gradient variational Bayes [2, 5, 1]. The key ingredient is an unbiased and low-variance way of estimating gradients with respect to distribution parameters from gradients evaluated at distribution samples. The “repar...
متن کامل